Enhanced capture, management and distribution of live presentations

ABSTRACT

Techniques are provided for converting live presentations into electronic media and managing captured media assets for distribution. An exemplary system includes capture devices that capture media assets of live presentations comprising a session, including image data of sequentially presented visual aids accompanying the live presentations and audio data. Each capture device has an interface for real-time image data marking of the image data for identification of individual images and session marking of the image data for demarcation of individual presentations of the session. A centralized device processes the captured media assets and automatically divides the captured media assets into discrete files associated with the individual presentations based on the session markings. An administrative tool manages the processed media assets to produce modified presentations and enables modification of the visual aid images identified by the image data markings. A production device formats the modified presentations for distribution on distribution media.

This application is a continuation of U.S. application Ser. No.12/749,215, filed Mar. 29, 2012, which is a continuation of U.S.application Ser. No. 11/580,092, filed Oct. 13, 2006, which is acontinuation-in-part of U.S. application Ser. No. 09/955,939, filed Sep.20, 2001, which is a continuation-in-part of U.S. Pat. No. 6,789,228,issued Sep. 7, 2004, all of which are herein incorporated by referencein their entireties. This application also claims the benefit of U.S.Provisional Application 60/726,175, filed Oct. 14, 2005, which is hereinincorporated by reference in its entirety.

BACKGROUND

1. Field of the Invention

The present invention generally relates to a data processing system fordigitally recording and reproducing lectures/presentations in physicaland electronic format. More particularly, the present invention relatesto the capture, management, and distribution of live presentations.

2. Related Art

The majority of corporate and educational institution training occurs inthe traditional lecture format in which a speaker addresses an audienceto disseminate information (i.e., a “live” presentation). Due todifficulties in scheduling and geographic diversity of speakers andintended audiences, a variety of techniques for recording the content ofthese lectures have been developed. These techniques include videotapes,audio tapes, transcription to written formats and other means ofconverting lectures to analog (non-computer based) formats, andconverting lectures to appropriate digital formats for use over theInternet.

A challenge arises with respect to capture and distribution of livepresentations at conferences and meetings, during which a large numberof lectures might be delivered over the course of one or several days,thereby making it difficult for a conference attendee to attend each ofthe lectures. Because such conferences have limited “shelf-life,” speedto market of the conference content is a critical element of success.Also important is the ability to accurately capture presentation contentfor distribution such that the captured content is precise in itspresentation and has the necessary speaker permissions (i.e., does notcontain information containing copyrighted materials without having thenecessary permissions associated with them). Additionally, as greaterquantities of presentation content are captured over relatively shortperiods of time, and as rapid release of the content becomesincreasingly important, effective management of the capturedpresentations is needed.

SUMMARY

Systems and methods are described herein that can be employed for rapidconversion of live presentations into electronic media and for effectivemanagement of captured media assets for distribution.

An exemplary system for capturing and distributing presentationsincludes at least two capture devices configured to capture media assetsof live presentations comprising a session, the media assetsincluding 1) image data of a plurality of sequentially presented visualaids accompanying the live presentations and 2) audio data. At least twoof the visual aids are selected from the group of images consisting ofslides, photographs, graphs, discrete motion picture clips, and text.Each capture device includes an interface that enables real-time 1)image data marking of the image data for identification of individualimages and 2) session marking of the image data for demarcation ofindividual presentations of the session. A centralized device isconfigured to process the captured media assets from each capturedevice. The centralized device is configured to automatically divide thecaptured media assets for the session into discrete files associatedwith the individual presentations based on the session markings. Anadministrative tool is configured to manage the processed captured mediaassets to produce modified presentations. The administrative toolenables modification of the visual aid images identified by the imagedata markings. A production device is configured to format the modifiedpresentations of at least one session for distribution on distributionmedia.

Another exemplary system for capturing and distributing presentationsincludes means for capturing media assets of live presentationscomprising a session, the media assets including 1) image data of aplurality of sequentially presented visual aids accompanying the livepresentations and 2) audio data. At least two of the visual aids areselected from the group of images consisting of slides, photographs,graphs, discrete motion picture clips, and text. The system includesmeans for real-time image data marking of the image data foridentification of individual images and means for real-time sessionmarking of the image data for demarcation of individual presentations ofthe session. Means for processing the captured media assets areconfigured to automatically divide the captured media assets for thesession into discrete files associated with the individual presentationsbased on the session markings. Means for managing the processed capturedmedia assets to produce modified presentations are configured to modifythe visual aid images identified by the image data markings. The systemalso includes means for formatting the modified presentations of atleast one session for distribution on distribution media.

An exemplary method for capturing and distributing presentationsincludes capturing media assets of live presentations comprising asession, the media assets including 1) image data of a plurality ofsequentially presented visual aids accompanying the live presentationsand 2) audio data. At least two of the visual aids are selected from thegroup of images consisting of slides, photographs, graphs, discretemotion picture clips, and text. The capturing includes real-time 1)image data marking of the image data for identification of individualimages and 2) session marking of the image data for demarcation ofindividual presentations of the session. The method also includesprocessing the captured media assets for the session. The processingincludes automatically dividing the captured media assets for thesession into discrete files associated with the individual presentationsbased on the session markings. The method further includes managing theprocessed captured media assets to produce modified presentations. Themanaging includes modifying the visual aid images identified by theimage data markings. Additionally, the method includes formatting themodified presentations of at least one session for distribution ondistribution media.

These and other features of the present disclosure will be readilyappreciated by one of ordinary skill in the art from the followingdetailed description of various implementations when taken in connectionwith the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates hardware components of a system consistent with thepresent disclosure;

FIG. 2 illustrates a mirror assembly used to redirect light from aprojection device to a digital camera consistent with the presentdisclosure;

FIG. 3 depicts the components of a computer consistent with the presentdisclosure;

FIG. 4 illustrates alternate connections to an overhead projector andLCD projector consistent with the present disclosure;

FIG. 5 shows input and output jacks on a system consistent with thepresent disclosure;

FIG. 6 is a flowchart illustrating a method for capturing a lectureconsistent with the present disclosure;

FIG. 7 is a flowchart illustrating a method for enhancing, a capturedlecture consistent with the present disclosure;

FIG. 8 is a flowchart illustrating a method for publishing a capturedlecture on the Internet consistent with the present disclosure;

FIG. 9 shows an example of a front-end interface used to access thedatabase information consistent with the present disclosure;

FIG. 10 shows a schematic of a three-tier architecture consistent withthe present disclosure;

FIG. 11 shows an alternative implementation consistent with the presentdisclosure in which the projection device is separate from the lecturecapture hardware;

FIG. 12 shows alternate connections to an overhead projector with amirror assembly consistent with the present disclosure;

FIG. 13 depicts the components of a embodiment for capturing a livepresentation where the images are computer generated;

FIG. 14 is a flow chart illustrating a method for capturing a lectureconsistent with an illustrated embodiment;

FIG. 15 depicts the components of another embodiment for use incapturing a live presentation in which the images are computergenerated;

FIG. 16 is a flow chart illustrating a method for capturing a livepresentation consistent with an illustrated embodiment;

FIG. 17 depicts the components of another embodiment for capturing livepresentations where the images are computer generated;

FIG. 18 is a flow chart illustrating a method for capturing a livepresentation consistent with an illustrated embodiment;

FIG. 19 depicts the components of another embodiment for capturing alive presentation where the images are computer generated; and

FIG. 20 is a flow chart illustrating a method for capturing a livepresentation consistent with an illustrated embodiment.

FIG. 21 illustrates an exemplary environment for conversion of livepresentations into electronic media consistent with embodiments of thepresent disclosure;

FIG. 22 depicts an exemplary user interface for a capture applicationaccording to an embodiment of the present disclosure;

FIG. 23 depicts an exemplary user interface for an editor tool accordingto an embodiment of the present disclosure;

FIG. 24 depicts an exemplary user interface for a server applicationaccording to an embodiment of the present disclosure;

FIGS. 25-28 depict exemplary user interfaces for an administrative toolaccording to an embodiment of the present disclosure;

FIG. 29 depicts an exemplary user interface for a production toolaccording to an embodiment of the present disclosure; and

FIG. 30 is a flow chart illustrating a method for capturing, managing,and distributing live presentations according to an embodiment of thepresent disclosure.

DETAILED DESCRIPTION

Systems consistent with the present disclosure digitally capture lecturepresentation slides and speech and store the data in a memory. They alsoprepare this information for Internet publication and publish it on theInternet for distribution to end-users. These systems comprise threemain functions: (1) capturing the lecture and storing it into a computermemory or database, (2) generating a transcript from the lecture and thepresentation slides and automatically summarizing and outliningtranscripts, and (3) publishing the lecture slides image data, audiodata, and transcripts on the Internet for use by Internet end-users.

In one implementation, slides can be generated using conventional slideprojectors. In this case, when a presenter begins presenting, and afirst slide is displayed on the projection screen by a projector, amirror assembly can change the angle of the light being projected on thescreen for a brief period of time to divert it to a digital camera. Atthis point, the digital camera captures the slide image, transfers thedigital video image data to the computer, and the digital video imagedata can be stored on the computer. The minor assembly then quicklyflips back into its original position to allow the light to be projectedon the projection screen as the presenter speaks. When this occurs, aninternal timer on the computer begins counting. This timer marks thetimes of the slide changes during the lecture presentation.Simultaneously, the system begins recording the sound of thepresentation when the first slide is presented. The digital images ofthe slides and the digital audio recordings are stored on the computeralong with the time stamp information created by the timer on thecomputer to synchronize the slides and audio.

Upon each subsequent slide change, the mirror assembly quickly divertsthe projected light to the digital camera to capture the slide image ina digital form, and then it flips back into its original position toallow the slide to be displayed on the projection screen. The time ofthe slide changes, marked by the timer on the computer, is recorded in afile on the computer. At the end of the presentation, the audiorecording stops, and the computer memory stores digital images of eachslide during the presentation and a digital audio file of the lecturespeech. Additionally, the computer memory also stores a file denotingthe time of each slide change.

Alternatively, in another implementation, slides can be generatedwithout using conventional slide projectors. For example, a computergenerated slide presentation can be used, thereby avoiding the need ofthe minor assembly and the digital camera. In this case, data fromapplication software, such as PowerPoint® (available from MicrosoftCorporation of Redmond, Wash.), or from any other application software apresenter is using to generate a presentation on his or her computer,can be captured. The digital video image data of a presentation slidefrom the presenter's computer can be transferred to the capture system'scomputer at the same time that the slide is projected onto theprojection screen. Similarly, slides may be projected from a machineusing overhead transparencies or paper documents. This implementationalso avoids the need for the minor assembly and the digital camera,because, like the computer generated presentations, the image data istransferred directly to the capture system's computer for storage at thesame time that the image data is projected onto the projection screen.Any of these methods or other methods may be used to capture digitalvideo image data of the presentation slides in the capture system'scomputer. Once stored in the computer, the digital video and audio filesmay be published to the Internet or, optionally, enhanced for moreefficient searching on the Internet.

During optional lecture enhancement, optical character recognitionsoftware can be applied to each slide image to obtain a text transcriptof the words on a slide image. Additionally, voice recognition softwarecan be applied to the digital audio file to obtain a transcript of thelecture speech. To enhance recognition accuracy, each presenter may reada standardized text passage (either in a linear or interactive fashion,in which the system re-prompts the presenter to re-state passages thatare not recognized in order to enhance recognition accuracy) into thesystem prior to presenting and, in doing so, provide the speechrecognition system with additional data to increase recognitionaccuracy. Speech recognition systems, which provide for interactivetraining and make use of standardized passages (which the presenterreads into the system) to increase accuracy, are available from avariety of companies including Microsoft, IBM and others. Oncetranscripts are obtained, automatic summarization and outlining softwarecan be applied to the transcripts to create indexes and outlines thatare easily searchable by an end-user. In addition to the enhanced files,the end-user can also search the whole transcript of the lecture speech.

Alternatively, if Closed Captioning is used during a presentation, theClosed Caption data can be parsed from the input to the device and atime-stamp can be associated with the captions. Parsing of the ClosedCaption data can occur either through the use of hardware (e.g., withClosed Caption decoder chips), such as those offered by PhilipsElectronics, or software, such as that offered by Ccaption (ccaption.comon the World Wide Web). The Closed Caption data can be used to provideindexing information for use in search and retrieval for all or parts ofindividual or groups of lectures.

In addition, information and data, which are used during the course ofpresentation(s), can be stored in the system to allow for additionalsearch and retrieval capabilities. The data contained and associatedwith files used in a presentation can be stored and this data can beused in part or in whole to provide supplemental information for searchand retrieval. Presentation materials often contain multiple media typesincluding text, graphics, video, and animations. With extraction ofthese materials, they can be placed in a database to allow additionalsearch and retrieval access to the content. Alternatively, the data canbe automatically indexed using products, which provide thisfunctionality, such as Microsoft Index Server or Microsoft PortalServer.

Finally, after transferring the files to a database, systems consistentwith the present disclosure can publish these slide image files, audiofiles and transcript files to the Internet for use by Internetend-users. These files can be presented so that an Internet client canefficiently search and view the lecture presentation.

Systems consistent with the present disclosure thus allow a lecturepresentation to be recorded and efficiently transferred to the Internetas active or real-time streaming files for use by end-users. The presentdisclosure therefore describes systems that are not only efficient atpublishing lectures on the Web, but can be efficient at recording thecontent of meetings, whether business, medical, judicial or another typeof meeting. At the end of a meeting, for instance, a record of themeeting complete with recorded slides, audio and perhaps video can bestored. The stored contents can be placed on a removable media such ascompact discs, digital versatile discs, flash memory, magnetic memory,or any type of recordable media to be carried away by one or more of themeeting participants.

Further, the present disclosure can be implemented as an effectiveteleconferencing mechanism. Specifically, so long as a participant in ateleconference has a capture device in accordance with the presentdisclosure, his or her presentation can be transmitted to otherparticipants using the recorded presentation, which has been convertedto a suitable Internet format. The other teleconference participants canuse similar devices to capture, enhance and transmit theirpresentations, or simply have an Internet enabled computer, Internetenabled television, wireless device with Internet access or likedevices.

These and further aspects of the systems and methods will be describedin the following sections. The explanation will be by way of exemplaryembodiments to which the present invention is not limited.

System Description

FIGS. 1 and 2 illustrate hardware components in a system consistent withthe present disclosure. Although FIG. 1 shows an implementation with aslide projector, the system allows a presenter to use a variety of mediafor presentation, such as, but not limited to, 35 mm slides, computergenerated stored and/or displayed presentations, overhead transparenciesor paper documents. Implementations using overhead transparencies andpaper documents will be discussed below with reference to FIG. 4.

FIG. 1 demonstrates an exemplary system with an integrated 35 mm slideprojector 100 that contains a computer as a component or a separateunit. The output of the projection device passes through an opticalassembly that contains a minor, as shown in FIG. 2. In theimplementation shown in FIG. 1, the mirror assembly 204 is contained inthe integrated slide projector 100 behind the lens 124 and is not shownon the FIG. 1. This minor assembly 204 diverts the light path to acharge-coupled device (CCD) 206 for a brief period of time so that theimage may be captured. A CCD 206 is a solid-state device that convertsvarying, light intensities into discrete digital signals, and mostdigital cameras (e.g., the Pixera Professional Digital Camera availablefrom Pixera Corporation of Los Gatos, Calif.) use a CCD for the digitalimage capturing process. The video signal carrying the digital videoimage data from the CCD 206, for example, enters a computer 102, whichis integrated within the projection box in this implementation, via adigital video image capture board contained in the computer (e.g., TARGA2000 RTX PCI video board available from Truevision of Santa Clara,Calif.). Naturally, the image signal can be video or a still imagesignal. This system can be equipped with a device (e.g., Grand TeleViewavailable from Grandtec UK Limited, Oxon, UK) that converts from an SVGAor Macintosh computer output signal into a format which can be capturedby the Truevision card, whereas the Truevision card accepts an NTSC(National Television Standards Committee) signal.

As the presenter changes slides or transparencies, the computer 102 canautomatically record the changes. Changes can be detected either by aninfrared (IR) slide controller 118 and IR sensor 104, a wired slidecontroller (not shown) or an algorithm driven scheme implemented in thecomputer 102 which detects changes in the displayed image.

As shown in FIG. 2, when a slide change is detected either via the slidecontroller 118 or an automated algorithm, the minor 208 of the minorassembly 204 is moved into the path of the projection beam at a45-degree angle. A solenoid 202, which is an electromagnetic deviceoften used as a switch, can control the action of the mirror 208. Thisaction directs all of the light away from the projection screen 114 andtowards the CCD 206. The image is brought into focus on the CCD 206,digitally encoded and transmitted to the computer 102 via thevideo-capture board 302 (shown in FIG. 3 described below). At thispoint, the mirror 208 flips back to the original position allowing thelight for the new slide to be directed towards the projection screen114. This entire process can takes less than one second, since the imagecapture is a rapid process. Further, this rapid process is not easilydetectable by the audience since there is already a pause on the orderof a second between conventional slide changes. In addition, the exacttime of the slide changes, as marked by a timer in the computer, can berecorded in a file on the computer 102.

FIG. 3 depicts the computer 102 contained in the integrated slideprojector 100 in this implementation. The computer 102 includes a CPU306 capable of running Java applications (such as the Intel Pentium(e.g., 400 MHz Pentium II Processors), central processors, and IntelMotherboards (IntelB N440BX server board) from Intel of Santa Clara,Calif.), an audio capture card 304 (e.g., AWE64 SoundBlaster™ availablefrom Creative Labs of Milpitas, Calif.), a video capture card 302, anEthernet card 314 for interaction with the Internet 126, a memory 316,and a secondary storage device 310. In one implementation, the secondarystorage device 310 can be a combination of solid state Random AccessMemory (RAM) that buffers the data, which is then written onto a CompactDisc Writer (CD-R) or Digital Versatile Disc Writer (DVD-R).Alternatively a combination or singular use of a hard disk drive, orremovable storage media and RAM can be used for storage. Using removablememory as the secondary storage device 310 enables participants to walkaway from a lecture or meeting with a complete record of the content ofthe lecture or meeting. Thus, neither notes nor complicated,multi-format records will have to be assembled and stored, and capturingthe actual contents of the lecture or meeting is made simple andcontemporaneous. Participant(s) can simply leave the lecture or meetingwith an individual copy of the lecture or meeting contents on a physicalstorage medium.

The computer 102 can also include or be connected to an infraredreceiver 312 to receive a slide change signal from the slide changecontroller 118. The CPU 306 can also have a timer 308 for marking slidechange times, and the secondary storage device 310 can contain adatabase 318 for storing and organizing the lecture data. The system canalso allow for the use of alternative slide change data (which isprovided as either an automated or end-user selectable feature) obtainedfrom any combination or singular use of: (1) a computer keyboard whichcan be plugged into the system, (2) software running on a presenter'spresentation computer which can send data to the capture device, or (3)an internally generated timing event within the capture device whichtriggers image capture. For example, image capture of the slide(s) canbe timed to occur at predetermined or selectable periods. In this way,animation, video inserts, or other dynamic images in computer generatedslide presentations can be captured, at least as stop action sequences.Alternatively or additionally, the slide capture can be switched to avideo or animation capture during display of dynamically changingimages, such as that which occurs with animation or video inserts incomputer generated slides. Thus, the presentation can be fully capturedincluding capture of dynamically changing images, with a potentialincrease in file size.

Referring back to FIG. 1, the computer 102 can include an integrated LCDdisplay panel 106, and a slide-out keyboard 108, which can be used toswitch among three modes of operation discussed below. For file storageand transfer to other computers, the computer 102 can also include afloppy drive 112 and a high-capacity removable media drive 110, such asa Jaz™ drive available from Iomega of Roy, UT (iomega.com on the WorldWide Web), among other devices. The computer 102 may also be equippedwith multiple CPUs 306, thus enabling the performance of several taskssimultaneously, such as capturing a lecture and serving a previouslycaptured lecture over the Internet.

Simultaneously with the slide capturing, audio signals can be recordedusing a microphone 116 connected by a cable 120 to the audio capturecard 304, which is an analog-to-digital converter in the computer 102,and the resulting audio files can be placed into the computer's 102secondary storage device 310, in this exemplary embodiment.

In one implementation consistent with the present disclosure, thepresentation slides are computer generated. In the case of a computergenerated presentation, the image signal from the computer (not shown)generating the presentation slides is sent to a VGA to NTSC conversiondevice and then to the video capture board 302 before it is projectedonto the projection screen 114, thus eliminating the need to divert thebeam or use the minor assembly 204 or the CCD 206. This also results ina higher-quality captured image.

FIG. 4 illustrates hardware for use in another implementation in whichoverhead transparencies or paper documents are used instead of slides orcomputer generated images. Shown in FIG. 4 is an LCD projector 400 withan integrated digital camera 402, such as the Toshiba MediaStar TLP-511U. This projection device allows overhead transparencies and paperdocuments to be captured and converted to a computer image signal, suchas an SVGA signal. This SVGA signal can then be directed to anSVGA-input cable 404. In this case, the computer 102 can detect thechanging of slides via an algorithm that senses abrupt changes in imagesignal intensity, and the computer 102 can record each slide change. Asin the computer generated implementation, the signal can be captureddirectly before being projected (i.e., the mirror assembly 204 and CCD206 combination shown in FIG. 2 is not necessary).

In one implementation, optical character recognition can be performed onthe captured slide data using a product such as EasyReader Elite™ fromMimetics of Cedex, France. Also, voice recognition can be performed onthe lecture audio using a product such as Naturally Speaking™ availablefrom Dragon Systems of Newton, Mass. The optical and voice recognitionprocesses can be used to generate text documents containing fulltranscripts of both slide content and audio of an actual lecture. Inanother implementation, these transcripts can be processed byoutline-generating software, such as LinguistX™ from InXight of PaloAlto, Calif., which can summarize the lecture transcripts, improvecontent searches and provide indexing. Other documents and informationcan then be linked to the lecture (e.g., an abstract, author name, date,time, and location) based on the content determination. The informationcontained in the materials (or the native files themselves) used duringthe presentation can also be stored into the database to enhance searchand retrieval through any combination or singular use of: (1) the datain a native format which is stored within a database, (2) components ofthe information stored in the database, and (3) pointers to the datawhich are stored in the database.

Most of these documents (except, e.g., those stored in their nativeformat), along with the slide image information, are converted toWeb-ready formats. This audio, slide, and synchronization data can bestored in the database 318 (e.g., Microsoft SQL) which is linked to eachof the media elements. The linkage of the database 318 and other mediaelements can be accomplished with an object-linking model, such asMicrosoft's Component Object Model (COM). The information stored in thedatabase 318 can be made available to Internet end-users through the useof a product such as Microsoft Internet Information Server (IIS)software, among others, and can be configured to be fully searchable.

Methods and systems consistent with the present disclosure thus enablethe presenter to give a presentation and have the content of the lecturemade available on the Internet with little intervention. Whileperforming the audio and video capture, the computer 102 canautomatically detect slide changes (i.e., via the infrared slide deviceor an automatic sensing algorithm), and the slide change information canbe encoded with the audio and video data. In addition, the Web-basedlecture can contain data not available at the time of the presentationsuch as transcripts of both the slides and the narration, and an outlineof the entire presentation. The presentation can be organized using bothtime coding and the database 318, and can be searched and viewed using astandard Java™ enabled Web-interface, such as Netscape Navigator™. Javais a platform-independent, object-oriented language created by SunMicrosystems™. The Java programming language is further described in“The Java Language Specification” by James Gosling, Bill Joy, and GuySteele, Addison-Wesley, 1996, which is herein incorporated by reference.In one implementation, the computer 102 can serve the lectureinformation directly to the Internet if a network connection 122 isestablished using the Ethernet card 314 or modem (not shown). Customsoftware, written in Java for example, can be used to integrate all ofthe needed functions for the computer.

FIG. 5 shows, in detail, the ports contained on the back panel 500 ofthe integrated 35-mm slide projection unit 100 consistent with thepresent disclosure: SVGA-in 502, SVGA-out 502, VHS and SVHS in and out510-516, Ethernet 530, modem 526, wired slide control in 522 and out524, audio in 506 and out 508, keyboard 532 and mouse port 528. Inaddition, a power connection (not shown) is present.

Operation

Generally, three modes of operation will be discussed consistent withthe present disclosure. These modes include: (1) lecture-capture mode,(2) lecture enhancement mode, and (3) Web-publishing mode.

(1) Capturing Lectures

FIG. 6 depicts a flowchart illustrating a method for capturing a lectureconsistent with the present disclosure. This lecture-capture mode can beused to capture lecture content in a format that is ready for publishingon the Internet. The system can create data from the slides, audio andtimer, and save them in files referred to as “source files.”

At the beginning of a lecture, a presenter prepares the media of choice(step 600). If using 35-mm slides, the slide carousel is loaded into thetray on the top of the projector 100. If using a computer generatedpresentation, the presenter connects the slide-generating computer tothe SVGA input port 502 shown in the I/O ports 500 of a projection unit100. If using overhead transparencies or paper documents, the presenterconnects the output of a multi-media projector 400 (such as the ToshibaMediaStar described above and shown in FIG. 4) to the SVGA input port502. A microphone 116 is connected to the audio input port 506, and anEthernet networking cable 122 is attached between the computer 102 and anetwork outlet in the lecture room. For ease of the discussion tofollow, any of the above projected media will be referred to as“slides.”

At this point, the presenter places the system into “lecture-capture”mode (step 602). In one implementation, a keyboard 108 or switch (notshown) can be used to set the lecture-capture mode. When this mode isset, the computer 102 can create a directory or folder on the secondarystorage device 310 with a unique name to hold source files for thisparticular lecture. The initiation of the lecture-capture mode can alsoreset the timer and slide counter to zero (step 603). In oneimplementation, three directories or folders can be created to hold theslides, audio and time stamp information. Initiation of lecture-capturemode can also cause an immediate capture of a first slide using theminor assembly 204 (step 604), for instance. The minor assembly 204flips to divert the light path from the projector to the CCD 206 of thedigital camera. Upon capturing the first slide, the digital image can bestored in an image format, such as a JPEG format graphics file (a Webstandard graphics format), in a slides directory on the secondarystorage device 310 of the computer 102 (e.g., slides/slide01.jpg). Afterthe capturing of the image by the CCD 206, the mirror assembly 204 flipsback to allow the light path to project onto the projection screen 114.The first slide is then projected to the projection screen 114, and theinternal timer 308 on the computer 102 begins counting (step 606).

Next, the audio of the lecture can be recorded through the microphone116 and the audio signal can be passed to the audio capture card 304installed in the computer 102 (step 608). The audio capture card 304converts the analog signal into a digital signal that can be stored as afile on the computer 102. When the lecture is completed, this audio filecan be converted into a streaming media format, such as Active StreamingFormat or RealAudio format, for efficient Internet publishing. In oneimplementation, the audio signal can be encoded into the ActiveStreaming Format or RealAudio format in real time as it arrives andplaced in a file in a directory on the secondary storage device 310.Although this implementation might require additional hardware (e.g., anupgraded audio card), it avoids conversion of the original audio fileinto Internet formats after the lecture is complete. Regardless, theoriginal audio file (i.e., unencoded for streaming) can be retained as abackup on the secondary storage device 310.

When the presenter changes a slide (step 610) using the slide control118 or by changing the transparency or document, the computer 102 canincrement the slide counter by one and record the exact time of thischange in an ASCII file (a computer platform and application independenttext format), referred to as a “time-stamp file,” written on thesecondary storage device 310 (step 512). This file can have, forexample, two columns, one denoting the slide number and the otherdenoting the slide change time. In one implementation, the file isstored in the time-stamp folder.

Using the minor assembly 204 (FIG. 2), a new slide can be captured intoa JPEG format graphics file (e.g., “slide#.jpg,” where # is the slidenumber) that can be stored in the slides folder on the secondary storagedevice 310. When the new slide is captured, the minor assembly 204quickly diverts the light from the slide image back to the projectionscreen 114 (step 616). If any additional slides are presented, theseslides can be handled in the same manner (step 618), and for eachadditional slide, the system can record the slide change time andcapture the new slide in the JPEG graphics file format.

At the completion of the lecture, the presenter, or someone else, canstop the “lecture-capture” mode with the keyboard 108. This action stopsthe timer and completes the lecture capturing process.

2) Enhancing Lecture Content

FIG. 7 depicts a flowchart illustrating a method for enhancing acaptured lecture consistent with the present disclosure. In oneimplementation, the “lecture enhancement mode” is entered when thelecture is complete, or contemporaneous with continued capture ofadditional lecture content, and the system has all or an initial set ofthe source files described above. In this mode, the system can createtranscripts of the content of the slides and the lecture, and canautomatically categorize and outline these transcripts. Additionally,the slide image data files may be edited as well, for example, to removeunnecessary slides or enhance picture quality.

Initially, optical character recognition (OCR) can be performed on thecontent of the slides (step 700). OCR converts the text on the digitalimages captured by the CCD 206 (digital camera) into fully searchableand editable text documents. The performance of the optical characterrecognition may be implemented by OCR software on the computer 102. Inone implementation, these text documents can be stored as a standardASCII file. Through the use of the time-stamp file, this file can bechronologically associated with slide image data. Further, ClosedCaption data (if present) can be read from an input video stream andused to augment the indexing, search and retrieval of the lecturematerials. A software based approach to interpreting Closed Caption datais available from Leap Frog Productions (San Jose, Calif.) on the WorldWide Web. In addition, data from native presentation materials canfurther augment the capability of the system to search and retrieveinformation from the lectures. Metadata, including the presenter's name,affiliation, time of the presentation and other logistical informationcan also be used to augment the display, search and retrieval of thelecture materials. This metadata can be formatted in XML (ExtensibleMarkup Language) and can further enhance the system through compliancewith emerging distance learning standards, such as Shareable CoursewareObject Reference Model Initiative (SCORM). Documentation regardingdistance learning standards can be found at, among other websites,elearningforum.com on the World Wide Web.

Similarly, voice recognition can be performed on the audio file tocreate a transcript of the lecture speech, and the transcript can bestored as an ASCII file along with time-stamp information (step 702).The system can also provide a system administrator the capability toedit the digital audio files so as to remove caps or improve the qualityof the audio using products such as WaveConvertPro (Waves, Ltd.,Knoxville, Tenn.).

Content categorization and outlining of the lecture transcripts can beperformed by the computer 102 using a software package such asLinguistX™ from InXight of Palo Alto, Calif. (step 704). The resultinginformation can be stored as an ASCII file along with time-stampinformation.

3) Web Publishing

FIG. 8 is a flowchart illustrating a method for publishing a capturedlecture on the Internet consistent with the present disclosure. Afterlecture capture or enhancement (step 800), the system may be set to“Web-publishing mode.” It should be noted that the enhancement of thelecture files is not a necessary process before the Web-publishing modebut simply an optimization. Also, note that for the Web-publishing modeto operate, a live Ethernet port that is Internet accessible should beconnected using the current exemplary technology. Standard Internetprotocols (i.e., TCP/IP) can be used for networking. In this mode, allof the source files generated in the lecture-capture mode, as well asthe content produced in the enhancement mode, can be placed in database318 (step 800). Two types of databases may be utilized: relational andobject oriented. Each of these types of databases is described in moredetail below.

Consistent with the present disclosure, the system can obtain atemporary “IP” (Internet Protocol) address from a local server on thenetwork node to which the system is connected (step 802). The IP addressmay be displayed on the LCD panel display 106.

When a user accesses this IP address from a remote Web-browser, thesystem (the “server”) can transmit a Java applet to the Web-browser (the“client”) via the HTTP protocol, a standard Internet method used fortransmitting Web pages and Java applets (step 804). The transmitted Javaapplet provides a platform-independent front-end interface on the clientside. The front-end interface is described below in detail. Generally,this interface can enable the client to view all of the lecture content,including the slides, audio, transcripts and outlines. This informationcan be fully searchable and indexed by topic (such as a traditionaltable of contents), by word (such as a traditional index in the back ofa book), and by time-stamp information (denoting when slide changesoccurred).

The lecture data source files stored on the secondary storage device 310can be immediately served to the Internet as described above. Inaddition, in one implementation, the source files may optionally betransferred to external Web servers. These source files can betransferred via FTP (File Transfer Protocol), again using standardTCP/IP networking, to any other computer connected to the Internet. Thesource files can then be served as traditional HTTP Web pages or servedusing the Java applet structure discussed above, thus allowingflexibility of use of the multimedia content.

Use of the Captured Lecture and the Front-End Interface

The end-user of a system consistent with the present disclosure cannavigate rapidly through the lecture information using a Java appletfront-end interface. This platform-independent interface can be accessedfrom traditional PCs with a Java-enabled Web-browser (such as NetscapeNavigator™ and Microsoft Internet Explorer™) as well as Java-enabledNetwork Computers (NCs).

FIG. 9 shows a front-end interface 900 consistent with the presentdisclosure.

The front-end interface can provide a robust and platform-independentmethod of viewing the lecture content and performing searches of thelecture information. In one implementation, the interface includes amain window divided into four frames. One frame can show the currentslide 902 and contain controls for the slides 904, another frame canshow the audio controls 908 with time information 906, and a third framecan show the transcript of the lecture 910 and scroll to follow theaudio. The fourth frame can contain a dialog box in which the user canenter search terms 912, a pop-up menu with which the user can selecttypes of media they wish to search, and a button that initiates thesearch. Examples of search methodologies include chronological, voicetranscript, slide transcript, slide number, and keyword, among others.The results of the search can be provided in the first three framesshowing the slides, the audio and the transcripts. In anotherimplementation consistent with the present disclosure, another windowcan be produced which shows other relevant information, such as relatedabstracts.

Description of the Database Structure

Before the source files generated in the lecture capturing process canbe published in a manner that facilitates intelligent searching, indexesto the source files should be stored in a database. The database canmaintain links between all source files and searchable information suchas keywords, author names, keywords in transcripts, and otherinformation related to the lectures.

Two methods for organizing a database that contains multiple types ofmedia (text, graphics and audio) include object-oriented and relational.An object-oriented database links together the different media elements,and each object contains methods that allow that particular object tointeract with a front-end interface. Any type of media can be placedinto the object-oriented database, as long as methods of how this mediais to be indexed, sorted and searched are incorporated into the objectdescription of the media.

The second method involving a relational database provides linksdirectly to the media files, instead of placing them into objects. Theselinks determine which media elements are related to each other (i.e.,they are responsible for synchronizing the related audio and slidedata).

FIG. 10 shows a schematic of an exemplary three-tier architecture 1000that can be used to store and serve the multimedia content to theend-user. As shown in FIG. 10, the database 318 can include part of thethree-tier architecture 1000. The database 318 (labeled as the “datatier”) can be controlled by an intermediate layer instead of directly bythe end-user's interface 1002 (labeled as the “client tier”). The clientis a computer running a Web-browser connected to the Internet. Theintermediate layer, labeled as the “application tier,” can providescalability (i.e., more servers can be added without bringing down theapplication tier), queuing (i.e., requests from the client can be queuedat the application tier so that they do not overload the database 318),and increased compatibility. Although the application tier and front-endare Java based, the database 318 can communicate with the applicationtier in any manner which maximizes performance. The method ofcommunication, protocols used, and types of databases utilized do notaffect the communication between the business logic and the front-end.

FIG. 10 also shows how the application tier includes a Main ProcessingUnit (MPU) 1004 and middleware 1020. On the MPU 1004 can reside thecustom Java code that controls query processing 1008, managestransactions 1010 and optimizes data 1012. Additionally, this code canperform OCR 1014 and voice recognition 1016 and encode the media 1018.The middleware 1020 can provide a link between the custom Java code andthe database 318. This middleware 1020 is presently available as variousmedia application programming interfaces (APIs) developed by SunMicrosystems, Microsoft, and others. The middleware 1020 can abstractthe custom Java code from the database 318.

The end-user or client can interact with the MPU 1004 within theapplication tier. In addition, information entering the database 318from the “lecture-capture mode” of the system can enter at theapplication tier level as well. This information can then be processedwithin the MPU 1004, passed through the middleware 1020, and populatethe database 318.

Further Embodiments

There are many different methods of implementing a system that performsfunctions consistent with the present disclosure. Several embodimentsare described below.

1) Separation of the Minor Assembly from the Projection Device andComputer

FIG. 11 depicts a lower-cost and more modular way of providing thelecture-capturing functionality involving the separation of the minorassembly 204 and CCD 206 from the projection device. In this embodiment,the minor assembly 204 and CCD 206 can be incorporated into a separateunit that snaps onto the lens of the 35-mm slide projector 1102. Asshown in FIG. 11, the minor assembly 204 and CCD 206 is connected byvideo cable 1104 to the computer 102, which sits in a separate box. Thisconnection allows the computer 102 to receive digital video image datafrom the CCD 206 and to control the action of the minor 204 via thesolenoid 202 (shown in FIG. 2). The infrared (IR) beam from the slidecontroller 118 signals a slide change to both the slide projector 1102and the computer 102. Both the infrared sensors on both devices areconfigured to receive the same IR signal so that the slide controller118 can control both devices. For instance, the slide projector 1102 maybe purchased with a slide controller 118, in which case the slideprojector 1102 will already be tuned to the same IR frequency as theslide controller 118. An infrared sensor in the computer 102 may bebuilt or configured to receive the same IR frequency emitted by theslide controller 118. Such configuration of an IR sensor tuned to aparticular frequency is well known to those skilled in the art.Additionally, a computer monitor 1110 can be used in place of the LCDdisplay on a single unit. A laptop computer, of course, can be usedinstead of the personal computer shown. This modular setup can providean advantage, in certain implementations, in that once the appropriatesoftware is installed, the user can use any computer and projectiondevice desired, instead of having them provided in the lecture-capturingbox described above.

For capturing computer-generated presentations, the minor assembly isnot used and the video signal and mouse actions from the user'sslide-generating computer pass through the capture computer before goingto the LCD projector. This configuration enables the capture computer torecord the slides and change times.

FIG. 12 shows another implementation using the connection of a separateCCD 206 and minor assembly 204, described above, to a standard overheadprojector 1200 for the capture of overhead transparencies. A video cable1202 passes the information from the CCD 206 to the computer 27. Agooseneck stand 1204 holds the CCD 206 and mirror assembly 204 in frontof the overhead projector 1200.

2) Slide Capture Trigger

With the use of a Kodak Ektapro Slide Projector (Kodak, Rochester,N.Y.), which can either be incorporated into device 100 or used as astand-alone slide projector 1102, another method of communicating thestatus of the slide projector to the computer 102 can use the P-Comprotocol (Kodak, Rochester, N.Y.). The P-Com protocol is communicatedbetween the slide projector and the computer 102 over an RS-232interface that is built into the Ektapro projector. The informationobtained from the projector can provide the computer 102 with the datasignaling that a slide change has occurred whereupon the computer canthen digitally capture the slide. This approach alleviates the need fordetecting signals from the infrared controller 118 and IR sensor 104 orthe wired slide controller.

3) Front-End Interfaces

Although the front-end interface described above is Java-based, if thevarious modes of operation are separated, other front-end interfaces canbe employed. For example, if lecture-capture is handled by a separatedevice, its output is the source files. In this case, these source filescan be transferred to a separate computer and served to the Internet asa website including standard HTML files, for example.

In another implementation, the front-end interface can also be aconsumer-level box which includes a speaker, a small LCD screen, severalbuttons used to start and stop the lecture information, a processor usedto stream the information, and a network or telephone connection. Thisbox can approach the size and utility of a telephone answering machinebut can provide lecture content instead of just an audio message. Inthis implementation, the lecture content can be streamed to such adevice through either a standard telephone line (via a built-in modemfor example) or through a network (such as a cable modem or ISDN).Nortel (Santa Clara, Calif.) provides a “Java phone” which can be usedfor this purpose.

4) Application Tier Implementation

The system described in the Main Processing Unit (1004) and theApplication Programming Interface (1020) can be programmed using alanguage other than Java, e.g., C, C++ and/or Visual Basic Languages.

5) Optical Assembly for Image Capture

Another implementation of the present disclosure can replace the mirrorassembly 204 with a beam splitter (not shown). This beam splitter allowsfor slide capture at any time without interruption, but reduces theintensity of the light that reaches both the digital camera and theprojection screen 114. If a beam splitter is used, redundancies can beimplemented in the slide-capturing stage by capturing the displayedslide or transparency, for example, every 10 seconds regardless of theslide change information. This approach can help overcome any errors inan automated slide change detection algorithm and allow fortransparencies that have been moved or otherwise adjusted to berecaptured. At the end of the lecture, the presenter can select fromseveral captures of the same slide or transparencies and decide whichone should be kept.

System Diagnosis

In one implementation consistent with the present disclosure, the usercan connect a keyboard and a mouse, along with an external monitor, tothe SVGA-out port 504. This connection can allow the user access to theinternal computer 102 for software upgrades, maintenance, and otherlow-level computer functions. Note that the output of the computer 102can be directed to either the LCD projection device or the LCD panel106.

Wireless Communications

In one implementation consistent with the present disclosure, thenetwork connection between the computer and the Internet can be madeusing wireless technology. For example, a 900 MHZ connection (similar tothat used by high quality cordless phones) can connect the computer 102to a standard Ethernet wall outlet. Wireless LANs can also be used.Another implementation can use wireless cellular modems for the Internetconnection.

Electronic Pointer

In another implementation, an electronic pointer can be added to thesystem. Laser pointers are traditionally used by presenters to highlightportions of their presentation as they speak. The movement of thesepointers can be tracked and this information can be recorded andtime-stamped. This approach can enable the end-user to search apresentation based on the movement of the pointer and have the audio andvideo portion of the lecture synchronized with the pointer.

Spatial positional pointers can also be used in the lecture-captureprocess. These trackers can allow the system to record the presenter'spointer movements in either 2-dimensional or 3-dimensional space.Devices such as the Ascension Technology Corporation pcBIRD™ or 6DOFMouse™ (Burlington, Vt.), INSIDETRAK HP by Polhemus Incorporated(Colchester, Vt.), or the Intersense IS 300 Tracker from Intersense(Cambridge, Mass.) can be used to provide the necessary trackingcapability for the system, among others. These devices send coordinate(x, y, z) data through an RS-232 or PCI interface, which communicateswith the CPU 306, and this data is time-stamped by the timer 308.

Separation into Different Units

In one embodiment consistent with the present disclosure, the system canbe separated into several physical units, one for each mode or a subsetcombination of modes (i.e., lecture capture, enhancement andpublishing). A first physical unit can include the projection device andcomputer that contains all of the necessary hardware to perform thelecture-capturing process. This hardware can include the minor assembly,the CCD digital camera, if this embodiment is used, a computer withvideo and audio capturing ability, an infrared sensing unit, andnetworking ability. In this implementation, the function of the firstphysical unit is to capture the lecture and create the source files onthe secondary storage of the unit. This capture device contains theprojection optics and can display one or more of 35-mm slides, acomputer generated presentation, overhead transparencies and paperdocuments.

In this implementation, the lecture enhancement activities can beperformed in a second separate physical enclosure. This separate devicecontains a computer with networking ability that can perform the OCR,voice recognition and auto-summarization of the source files generatedin the lecture-capturing process.

Finally, a third physical enclosure can provide Web-publishing functionand contain a computer with network ability, a database structure andInternet serving software. The second and third functions can becombined in one physical unit, the first and third functions can becombined in one physical unit or the first and second functions can becombined in one physical unit, as circumstances dictate.

In this modular design, several categories of products can beimplemented. For example, one implementation can provide lecturecapturing ability only, and require only the lecture-capturing devices.This system would be responsible for the creation and serving of thegenerated source files. Another implementation can provide lecturecapturing and Web serving, and only require the lecture-capturingdevices and the Web-publishing devices. Yet another implementation canadd the lecture-enhancement device to the above configurations, and alsoprovide the lecture transcripts and summaries to the Web. In addition tothe modularization of the different tasks as described above,modularization with respect to physical components (different products),with distributed task functions, can be achieved. For instance, severallecture capture units can be networked or otherwise connected to acentralized enhancement and publishing, or just publishing unit.

Electronic Capture Embodiments

The modular approach can facilitate additional embodiments where thepresentation is developed, at least regarding the slides, as acomputer-generated presentation using available software such asPowerPoint®, etc. In these embodiments, a chip set, such as that madeavailable from PixelWorks and other companies, can be employed to enableauto-detection of the video signal and also to provide digitization ofthe signal in a means which is appropriate to the resolution and aspectratio and signal type (video vs. data). The CPU and the digitizationcircuitry can be provided on a single chip along with a real-timeoperating system and Web-browser capability, or on separate chips. Fourembodiments with varying degrees of modularity and functionality aredescribed below. Furthermore, PixelWorks offers chip sets which providea system on a chip by incorporating a Toshiba general purposemicroprocessor, an ArTile TX79, on the same chip as the video processingcircuits (pixelworks.com/press on the World Wide Web). Leveraging thegeneral purpose microprocessor, embodiments containing this or similardevices can be configured to perform the following functions:

-   -   Control and/or communicate with external devices such as hard        drives or other digital storage media using USB, Ethernet and or        IEEE 1394 connectivity.    -   Execute software which can either read file formats (such as        Microsoft PowerPoint®, Microsoft Word®, Internet browsers, etc.)        which are commonly used in presentations.    -   Execute software to read a file in an intermediate file format        which may be a proprietary “transfer format,” which is        compatible with Microsoft PowerPoint®, Word, Internet browsers,        and the like commonly used in presentations. Companies that        produce such file translation software include DataViz        (dataviz.com on the World Wide Web).    -   Interpret data from an input stream (provided for example by        IEEE 1394, USB, Ethernet, or Wireless network connectivity),        allowing processing of data for either immediate display and/or        storage in part or in whole for later viewing.

1) Projector Embodiment

The first of these embodiments, shown in FIG. 13 includes a standardimage (e.g., slide and/or video) projector 1302 with an intermediaryunit 1370 placed between the projector 1302 and the source of theprojected images, e.g., a general purpose computer 1350. Theintermediate unit 1370 can complete the media processing and containeither a USB port 1374 to communicate with the computer 1350 andpossibly an analog modem and Ethernet to communicate directly with aserver 1390. The projector 1302 associated with this embodiment can beany commercial or proprietary unit that is capable of receiving VGA,SVGA, XGA or SXGA and/or a DVI input, for instance. The input 1305 tothe video projector is received via cable 1304 from the intermediateunit 1370 from an associated output port 1371. The intermediate unit1370 receives its input at interface 1372 via cable 1303 from thegeneral purpose computer 1350 or other computer used for generating thepresentation. The intermediate unit 1370 can also contain anomni-directional microphone 116 and audio line input to be usedconcurrently or separately as desired by the user. The intermediate unit1370 can function to capture the presentation through the computergenerated slides, encode time-stamp information and capture the audioportion of the presentation. The captured data can then be stored inremovable media 1380 or transferred via USB or other type of port fromthe intermediate units output 1372 by cable 1373 b to the computer 1350.This aspect can eliminate the need for storage in the intermediate unit1370 and can use more reliable flash memory. The computer 1350 or othertype of computer can receive the processed media from the intermediateunit 1370 and transfer the data via cable 1373 a to the Web-serverthrough its connection to the net. Optionally, the intermediate unit1370 can connect directly to the media server 1390 via cable 1373 a, asdescribed earlier.

The media server 1390 running standard media server software such asApple Quicktime™, RealNetworks RealSystem Server™ or Microsoft MediaServer, can stream the data with a high bandwidth connection to theInternet. This process can occur both as a simulcast of the lecture aswell as in an archive mode with transfer occurring after the event hastranspired. Such arrangement with the computer 1350 can eliminate theneed for an Ethernet card and modem built into the intermediate unit1370 since most general purpose computers already have thisfunctionality.

FIG. 14 shows a flow chart with each function arranged in an associatedcomponent, including a general purpose computer or other type ofcomputer 1350, an image projector 1302 and an intermediate unit 1370. Atthe beginning of a presentation, the presenter can use the computer 1350to send a computer generated presentation, i.e., an image or series ofimages or slides, to the intermediate unit 1370 in step 1401.Simultaneously with this process, the intermediate unit in step 1410begins to record the audio portion of the live presentation. In step1402, in the intermediate unit 1370, a signal containing the image issplit into two signals, the first of which is processed with therecorded audio in step 1406 and is stored in step 1407 in theintermediate unit 1370, or alternatively sent directly to the server instep 1408. In step 1403, the second of the split signals is sent to theprojector in step 1403, and is displayed by the projector 1302 in step1404. The process begins again at step 1401 when the lecture sends a newcomputer generated image. The audio is recorded continuously until thepresentation is complete.

In splitting the image signals sent from the personal computer 1350 atstep 1401, the present embodiment can facilitate two different methods.In the first method using an image signal splitter (e.g., a Bayview50-DIGI, see on the World Wide Web baytek.de/englisch/BayView50.htm),the image signal is split into a digital 24 bit RGB (red, green, blue)for media processing and an analog RGB image signal sent to theprojector 1302. However, if the projector is capable of receivingdigital RGB image signals, then an image signal splitter such as aBayview AD1 can be used, which produces two digital outputs, one forprocessing and one for projection.

2) Digital Output Projector

While an objective is to employ a standard, non-customized computer 1350to permit a presenter to use his own laptop, for instance, it ispossible that the functions of the intermediate unit 1370 can beincorporated in the general purpose computer 1350 through software,firmware and hardware upgrades.

In a second embodiment, such as shown in FIG. 15 for use with computergenerated presentations, an image projector 1502 can contain a digitaloutput and formatting for output via USB or Firewire (IEEE 1394). Ageneral purpose personal computer 1550 or other type of computer usedfor generating the presentation can supply the computer generatedpresentation to the projector 1502 through an input port 1505 via cable1505 a on the projector that has the capability of receiving VGA, SVGA,XGA or SXGA and/or a DVI input for instance. Through the USB or Firewire(IEEE 1394 interface) interface 1506, via cable 1505 a, the projector1502 can communicate with an intermediate unit 1570 at interface 1572,which can capture the computer generated presentation, as well as theaudio portion of the presentation through an omni-directional microphone116 and/or audio input. The output from the intermediary unit 1570 is ina raw media format and supplied to the general purpose computer 1550 viaUSB or Firewire interface 1571 and cable 1571 a, where the raw media canbe processed using custom software for media conversion and processingor custom hardware/software in the laptop computer. The media can beprocessed into HTML and/or streaming format via the software/hardwareand supplied to the media server 1590 via cable 1590 a, which in turncan stream the media with high bandwidth to the Internet 1500. Thissystem utilizes the capabilities of the computer 1550 used in generatingthe presentation to process the media, with the addition of software orsome custom hardware. The intermediate unit 1570 can also have aremovable storage media 1580 and presentation capture controls 1575capable of adjusting certain parameters associated with the lecturecapture. However, the intermediate unit 1570 can be connected directlyto the server 1590.

FIG. 16 is a flow chart representing different functions and componentsof the lecture capturing system for the embodiment shown in FIG. 15 anddiscussed above. At the start, the presenter via the computer 1550 sendsa computer generated presentation, e.g., images, to the projector atstep 1601. As in the previous embodiment, the image signal is split atstep 1602 into two image signals, the first of which is formatted, ifnecessary, to digital form which also can be carried out using thesignal splitting components discussed above. The signal is then storedat step 1606, along with the audio portion of the live presentationwhich is recorded in step 1609. The raw data is then transferred back tothe computer 1550 for media processing in step 1607 wheresynchronization of the recorded audio portion and the images is alsoaccomplished. The formatted information is then sent to a server in step1608.

3) Projector with Media Processor

In a third embodiment for use with computer generated presentationsshown in FIG. 17, the projector 1702 contains digital output andformatting for output via USB or Firewire and further contains the mediaprocessor, which can process the media into HTML and/or streaming formator other Internet language. The projector 1702 can communicate with amedia server 1790 through an Ethernet interface 1706 via cable 1706 a,from which the media can be streamed to a connection to the Internet1700. Again, this system can produce a simulcast of the lecture, as wellas store the lecture in an archive mode. This embodiment, as with theprevious embodiments, can allow the use of removal media 1780 in theprojector 1702. The projector 1702 can also contain a control panel 1775for controlling various parameters associated with capturing thepresentation. Optionally, the control panel can be created in softwareand displayed as a video overlay on top of the projected image. Thisoverlay technique is currently used on most video and/or data projectorsto adjust contrast, brightness and other projector parameters. Thesoftware control panel can thus be toggled on and off and controlled bypressing buttons on the projector or through the use of a remote controlwhich communicates with the projector using infrared or radio frequencydata exchange.

FIG. 18 is a flow chart showing the different functions and componentsof the live presentation capture system for the embodiment shown in FIG.17 and discussed above. The individual components in this embodimentinclude a computer 1750, a projector 1702 and a network server 1790. Atthe start of the presentation, the presenter using a laptop computersends a computer generated presentation, i.e., image, to the projector.The image signal is then divided in step 1802, as discussed previously,with one signal being used to project the image in step 1803, and theother signal being processed along with the audio portion of the livepresentation that was recorded at step 1808, in step 1804. The processedmedia can then be stored using fixed memory or removable memory media instep 1805. As discussed above, processed media can also be directly sentto the server 1790 through step 1806 without implementing the storagestep 1805. The server 1790 in step 1807 connects to the network orInternet such that it can be accessed by a client.

4) Projector with Enhancement and Publishing Capabilities

A fourth embodiment associated with computer generated presentations, asseen in FIG. 19, includes a projector 1902 that contains all thehardware necessary to capture and serve the electronic content of thelive presentation through a connection 1906 to the network throughEthernet or fiber connection. As such, the projector 1902 can capturethe video content through its connection via interface 1905 and cable toa personal computer 1950, or other type of computer. The projector 1902can capture the audio content via omni-directional microphone 116 oraudio line input. The projector 1902 can also process the media intoHTML and/or streaming format and further act as a server connectingdirectly to the Internet 1900. The projector 1902 can also contain acontrol panel 1975, which controls various parameters associated withcapturing the presentation, as well as removable media 1980 when it isdesired to store the presentation in such a manner.

FIG. 20 is a flow chart showing the functions and components used tocapture a live presentation according to the above embodiment shown inFIG. 19. At the start of the presentation the presenter, using thecomputer 1950, can send a computer generated presentation to theprojector 1902. Again, as discussed in detail above, after step 2001 thedata from the image signal is split into two signals in step 2002, thesecond signal being used to project the image in step 2003 such that itcan be viewed by the audience. The first signal is processed andsynchronized with the audio portion of the live presentation which wasrecorded in step 2007, in step 2004. The processed media can then bestored in step 2005 and/or streamed directly to the Internet step 2006.With the functions integrated all into one projector 1902, the projector1902 can be capable of functioning as each of the individual components,and such various interfaces and capabilities can be incorporated intothe projector.

Various inputs associated with a standard projector can be incorporatedinto the integrated projector, including, but not limited to, digitalvideo image and/or VGA. Outputs allowing the integrated projector tofunction with a standard projector, thus expanding its versatility, canalso include a digital video image output for highest quality digitalsignal to the projector. VGA output can also be integrated into theintegrated projector. USB connectors, as well as Ethernet and modemconnectors, an audio input and omni-directional microphone can also beincluded in the integrated projector 1902. As the integrated projector1902 is capable of many different functions using different sources,input selection switches can also be included on the integratedprojector, as well as other features common in projectors such as remotecontrol, and a variety of interfaces associated with peripheralelements.

The previous four embodiments employ similar processes for the captureof the presentation. In general, the presenter (or someone else)connects the personal computer (e.g., laptop) to the integratedprojector or the in-line of the intermediate unit. The system isconfigured, through available switches, depending on the source, tocapture characteristics unique to the source of the presentation. Theaudio is captured and converted to digital through an A and D converter,along with the images if the digital output from the projector is notavailable. The image signal is split and the image is displayed and thencompressed into a standard file format, (e.g., JPEG, MPEG). Thesynchronization of audio and images can occur during the digitizationand formatting processes. The media processing can include compressionof images via a variety of methods, including color paletteoptimization, imagery sizing and image and audio compression, as well asindexing. Compression for use of the data in an Internet stream formatcan also occur during processing. During media processing, other datacan also be entered into the system, such as speaker's name, title ofthe presentation, copyright information and other pertinent information,as desired. The information captured can then be transferred to theserver, allowing it to be streamed to clients connected to a network,Internet or Intranet. As discussed in the above embodiments, the mediacan be served directly from one of the intermediate units or projectors,or it can be transferred to an external server, which exists as part ofan Internet or is directly connected to the Internet. When the data ismade available immediately over an IP connection in either auni-directional or bi-directional manner, the device can be used forreal-time teleconferencing. As such, these embodiments are compatiblewith other methods and systems for capturing a live presentation, asdiscussed earlier, and as such can include other applicable featurespresented in this disclosure, as appropriate. More or lessmodularization of the system can be employed in response to varyingneeds and varying user assets.

5) Use of Digital Media with Embedded Processor/Operating Systems

Another embodiment involves the use of digital media which containmicroprocessors and independent operating systems. One representativedevice, the Mine from Teraoptix (mineterapin.com/terrapin on the WorldWide Web) contains the Linux operating system, digital storage (12gigabytes of storage) and Ethernet, USB, and IEEE 1394 connectivity.This device also allows for Internet connectivity for file uploads anddownloads. Coupling this device with the different embodiments can allowfor a solution which provides (or replicates) the digital audiorecording functionality, as well as provides image storage throughconnection of the projector which may be equipped with a USB, Ethernet,or IEEE 1394 output).

6) Software-Based Capture Embodiment

The laptop or presentation computer, in parallel with running thepresentation, can capture the presentation. The following components canbe employed to affect lecture capture in a software-based solution andare each described below in further detail:

-   -   i. Generation of time-stamps;    -   ii. Visual media processing;    -   iii. Audio capture and processing;    -   iv. Synchronization of media;    -   v. Addition of search methodologies to on-line presentations;        and    -   vi. Placement of materials on the Web and use of emerging        distance learning standards.

The software involved in the capture process is referred to herein asthe “capture application” (CA). The CA can run on the presentationsystem or on the server (or can partially run on both). The software canbe written in standard personal computer programming languages such asC, C++, JAVA, or other software languages.

Several approaches can be employed for (i) generation of time-stamps,including:

-   -   a. Use of the Microsoft COM protocol. When the presentation        makes use of applications which support COM (e.g., the Microsoft        Office Suite), the applications can communicate back to the CA        all of the operations and functions (events) which were        preformed using the application during a presentation. By        associating each event with a corresponding time-stamp, the CA        can create a time-line of events associated with the media,        allowing for the storage and transmission of a presentation.    -   b. Use of digital audio to generate time-stamp data. Events        during a presentation can be punctuated by changes in a        presenter's audio. For example, a presenter may pause between        the presentations of different media elements and/or the        presenter's speech may change in pitch at the end of the display        of a media element. Furthermore, the presenter may use ‘cues’        which signal changes in media (such as a statement, ‘on the next        slide’). Through signal processing techniques and/or speech        recognition, these events can be abstracted to create a        time-stamp/event log.    -   c. Use of changes in the visual elements. Through the use of        digital image processing software, time-stamp data can be        created. The digital image processing techniques can identify        movement of the pointer (associated with mouse movement) over        particular regions of the image, indicating changes in the        presentation. Other techniques involve changes in color palette        of images, and/or image file size.    -   d. Monitoring keyboard and mouse functions. Through the use of        software which provides a time-stamp when an event occurs, such        as mouse clicks, movement, as well as keyboard key depression, a        time-stamp log can be created.    -   e. PowerPoint slides presentations. Existing PowerPoint        presentations can be opened using an appropriate version of        Microsoft PowerPoint. The software can then provide the ability        to capture the PowerPoint presentations for broadcast on the        Internet. This functionality can further allow for the        conversion of the presentation into a Microsoft Media Player        format.    -   f. Any combination of the above techniques.

With each of the above time-stamp generation approaches, thepresentation computer can initiate capture either locally on thepresentation machine itself and/or on the server.

Several approaches can be employed for (ii) visual media processing. Forexample, the following approaches for image capture on the presentationcomputer can be implemented in singular or in combination.

-   -   a. Local Capture of Presentation Images. An example of local        image capture makes use of software techniques deployed by        companies such as TechSmith for screen capture (techsmith.com on        the World Wide Web), which can capture images through the use of        trigger events or on a timed basis.    -   b. Capture of Images through File Conversion. Optionally, the        native files used during a presentation can be converted into        Web-ready formats (e.g., JPEG) on the presentation machine,        server, or any intermediary device containing a microprocessor.    -   c. Video Capture. Use of a Web cam (such as produced by 3Com) or        other digital video source with a standard computer interface        (e.g., USB, IEEE 1394) can provide imaging of the presenter,        which can be combined with the presentation.

Several approaches can be employed for (iii) audio capture andprocessing, including the use of audio capture technology available onmany computers, in either hardware that exists on the motherboard orthat is provided with the addition of a digital audio acquisition cardfrom suppliers, such as Creative Labs. Optionally, a microphone whichconverts the audio signal into a digital format, such as USB availablefrom HelloDirect (hellodirect.com on the World Wide Web), can beconnected to the PC to enable audio capture. Audio capture software cancapture the audio into memory, hard-drive, removable storage, ortransmit directly to a server through the use of TCP-IP protocols ordirect connection through standard data cables such as USB or IEEE 1394cabling. After capture, the audio can either be stored in a variety ofstandard audio formats (e.g., MP-3, MP-2, AIFF, WAVE, etc.) or directlyinto a streaming format such as QuickTime, or RealNetworks streamingformats, among others.

A device, such as the Mine from Teraoptix Mine, can be used to augmentdigital audio capture and/or Internet connectivity. For example,software written in C, Java, or other programming languages, which isstored and executed on the Mine device, can record the digital audio onthe Mine device while communicating with the presentation personalcomputer. This communication can involve a standardized time generationwhich is used to generate the time-stamps during the presentation. As aresult, this system can segment the audio recording and time-stampingfunctionality to the Mine device and the image capture occurring on thesystem being used for the presentation.

Several approaches can be employed for (iv) addition of searchmethodologies to on-line presentations. For example, enhanced searchcapabilities can be created through the use of speech recognition, aswell as optical character recognition, abstraction of text, and otherdata and their use in a searchable database (as described above).Metadata can also be used for indexing and search and retrieval.

Several approaches can be employed for (v) placement of materials on theWeb and use of emerging distance learning standards. For example,integration of the media and its presentation on the Web can be enabledby transmitting the captured audio, visuals, and time-stamp information,along with other available data (including speech recognition format,closed caption data), obtained as described above. The additional searchmethodologies, as well as support of distance learning standardsdescribed above, can be applied to this embodiment. This data can beplaced on a server and made available to end-users over a network (e.g.,Intranet, Internet or Wireless Internet network). Alternatively, thepresentation can be placed on a removable media such as a CD-ROM or DVDfor distribution.

Enhanced Capture, Management and Distribution of Live Presentations

A detailed description of an exemplary hardware-software platformenabling rapid conversion of live presentations into electronic mediaand effective management of captured media assets for distributionfollows. The electronic media can be transmitted over the Internet usinga variety of audiovisual technologies. These technologies can offer arange of media formats, from full-motion video to visual-only oraudio-only formats, as described above. Exemplary streaming videoformats include Macromedia Flash®, Microsoft Media 9 or 10 or O1®formats, and Apple QuickTime®, among others. Exemplary audio formatsinclude MP-3 and podcasts, among others. An exemplary slide and audioformat includes Macromedia Flash®. Text-based formats include PDF andother text/graphic formats. In addition, the transmission and storage ofthese media materials can range from electronic formats to actualphysical formats (e.g., DVD and CD or monographs).

Effective management of the captured presentations is needed to handlegreater quantities of content captured over relatively short periods oftime, and to enable rapid release of the content for distribution. Forexample, effective presentation management techniques can be implementedto appropriately determine when a presenter begins and ends apresentation, to assign metadata associated with the presentation to thecaptured digital media (audio/visuals) for the presentation, to createenhanced navigation tools (e.g., thumbnails and additional metadata,which can be time-based), to provide editorial tools for replacement ofcopyrighted or other undesired information in a presentation, and toenhance the presentation with additional information (e.g., a standardtitle at the beginning of each presentation and/or a title specifying asponsor of a lecture, as well as when the question and answer portion ofa presentation began, etc.).

FIG. 21 illustrates an exemplary environment 2100 for conversion of livepresentations into electronic media consistent with embodiments of thepresent disclosure. The environment 2100 includes a hardware/softwarecapture platform 2105, data sources 2120 (e.g., metadata, video,computer presentations, etc.), a communications network 2150, supporteddistribution media 2155 (e.g., kiosk, Internet websites, CD/DVD, PDA,etc.), and interfaces for additional services 2160 (e.g., humanresources, accounting, help desk, authoring, etc.).

In the embodiment of FIG. 21, the hardware/software capture platform2105 includes a capture device 2110, a web server 2130, a database 2135,a centralized device 2125, a testing and certification tool 2140, and areporting server 2145. Hardware/software capture platform 2105, as wellas other embodiments of the present disclosure, can support distributionof live presentations, on different media for use on differentplatforms, as well as support on-line testing and certification.

1) Rapid Capture of Content

A large number of presentations are typically delivered over the courseof several days at a session, such as a conference or meeting, making itdifficult for an attendee to be present at each of the livepresentations. Thus, recognizing the need to simultaneously recordmultiple live presentations of a session, digital capturing devices,such as those disclosed in U.S. Pat. No. 6,789,228, can be used tocapture presentation content with simultaneous audio and video. Video ofthe data portion of a presentation, including image data of sequentiallypresented visual aids, can be input to a capture device, such as theENCORE capture device from Astute Technology, using VGA or DVI. In someembodiments, the resolution of the video images can be preserved byusing a frame rate of capture of approximately 12-15 frames per second.The capture device can be configured to capture dynamic data embedded ina presentation, such as movies, animations, transitions, etc., as wellas electronic pointer position data.

For example, the hardware/software capture platform 2105, shown in theembodiment of FIG. 21, includes an ENCORE capture device 2110, which canbe configured to capture media assets of live presentations of asession. The media assets can include image data of sequentiallypresented visual aids accompanying the live presentations, as well asaudio data. Exemplary visual aids include images of slides, photographs,graphs, discrete motion picture clips, and text accompanying the livepresentations.

The capture device can also be configured to enable switchable sources,for simultaneous capture of NTSC video (e.g., talking head), as well asthe selection of other VGA/DVI or video sources into the captured mediafile. For example, in one embodiment, the image data for a presentationcan include the visual aid images the presenter displayed during thepresentation, as well as video images of the presenter during thepresentation. In addition, real-time transition effects can be included,as well as title information and lower-third information (such as thename of a presenter) shown in the alpha channel during the capture andtransmission processes. The ability to select and switch content sourcescan further be enabled through custom code extensions using, forexample, Microsoft DirectShow® technology for capturing and encoding themedia files. In one implementation, the media assets can be simulcastduring archival capture.

Furthermore, the capture device can be configured to simultaneouslycapture high resolution image data, which can be used to augment thepresentation. These high resolution images can be used for OCR, as wellas to enable close-up views of content.

Each capture device can include an interface that enables real-timeimage data marking of the image data for identification of individualimages and session marking of the image data for demarcation ofindividual presentations of the session. For instance, FIG. 22 depictsan exemplary user interface 2200 for a capture application that can beexecuted on a capture device in accordance with an embodiment of thepresent disclosure. As shown in the embodiment of FIG. 22, the captureapplication includes a capture control window 2205 and a Windows MediaEncoder® (WME) window 2210. Using various fields in the capture controlwindow 2205, a user can select an appropriate session 2215 (or“unidentified session” if the session is not known) and correspondingpresentation/lecture information 2220, as well as a capture operator2225 and a capture station 2230 (which can be a preset field). Next, theuser can initiate capture of a presentation by selecting a “StartRecording” button 2240. Note that at this point, the WME window 2210does not need to be used to change preset capture parameters.

Once capture of a presentation is initiated, the user can set markers atany point during the capture process to mark a particular event. Forexample, marker buttons 2235 can be used to mark “in” the beginning andmark “out” the end of the presentation, as well as to set randommarkers. Notes can be associated with the markers (e.g., the user cantype in a note after setting a marker), which can be helpful in thesubsequent editing phase. At the end of the presentation, the user canselect the “Stop Recording” button 2245 to stop the capture process.Note that the WME window 2210 can typically take a few seconds to stopas well. The user can then close the capture application.

FIG. 23 depicts an exemplary user interface 2300 for an editorapplication that can be executed on the capture device or on thecentralized device according to embodiments of the present disclosure.The editor application can be configured to allow for selection of acaptured media file via a “Select Recorded File” dialog box 2305. In oneimplementation, the captured media assets can be organized by date andsession ID. After a user selects a session, the editor application canbe configured to automatically display the session file and the markersthat have been set for the session in the “Markers” window 2310. Theeditor application can also display in the “Conference Data” portion2315 of the interface 2300 any associated information that has beenstored in a database for the session.

A “Media Editor” preview window 2320 can be used to play back an entirerecording for a captured session and adjust the markers for theindividual presentations of the session. For example, when a marker inthe “Markers” window 2310 is selected, a video progress slider 2325 canautomatically advance to the proper position, highlighting the beginningand the end of the selected segment (i.e., presentation). The markerscan be adjusted as needed, and the start and end points of the segmentcan be reset, if necessary. Additionally, one second and ten secondbuttons 2330 can help the user navigate through the video of theselected segment.

When final markers are set, the “Confirm Lecture” button 2335 can beselected to confirm the selected segment as an individualpresentation/lecture, thereby associating a specific presenter/speakerfrom the “Lectures” window 2340 with the confirmed segment. Onceconfirmed, the status of the lecture/presentation can be changed from“Marked (M)” to “Markers Confirmed (MC)” by selecting the “ChangeStatus—Add Notes” button 2345. Optionally, the editor application can beconfigured to display a warning if the user attempts to overwriteexisting markers for a specific speaker/lecture. Additionally, an“Auto-advance” checkbox 2350 can be selected to automatically advance toa next lecture segment after having confirmed a selected lecturesegment. This editing process can be repeated until everylecture/presentation in the session is confirmed.

In an embodiment, the capture technology described herein can beemployed for behavior monitoring of end-users. For example, a view of auser as they are interfacing with a computer can be captured. In thisway, developers can analyze the user's expressions and interactions withthe computer interface to implement ergonomic modifications to thecomputer interface.

2) Rapid Enhancement of Content

Because it can be cumbersome and time consuming to edit a captured mediafile (e.g., to divide the media file into portions to incorporate only aspecific speaker's presentation or to edit a presentation), ahigh-quality, rapid technique for producing digital archives/librariesof content is provided. A centralized device can be configured toprocess the captured media assets from each capture device. In oneembodiment, the centralized device can automatically divide the capturedmedia assets for a session into discrete files associated with theindividual presentations of the session based on the session markings,which can be set and confirmed using the exemplary capture and editorapplications shown in FIGS. 22-23. For example, the hardware/softwareplatform 2105, shown in the embodiment of FIG. 21, includes acentralized device 2125, which can be configured to process the capturedmedia assets from each capture device 2110.

The centralized device can be used to rapidly specify when a givenspeaker is presenting during a presentation (i.e., the start and stoppoints in time), as well as to associate metadata (e.g., the speaker'sname, title of the presentation, etc.) and other supplemental materialswith the presentation. This information can then be associated with therecorded media files. After the edit “in” and “out” points of aparticular lecture/presentation are specified, the centralized devicecan be used to split the captured audio/visual files into discrete mediafiles, which are associated with particular lectures. In addition, ascene/slide detection technique can be employed to determine changes inthe sequence of a presentation to create unique, thumbnail images of thepresentation, thereby enhancing navigation.

3) Post Processing

After a session of presentations is captured and marked, the centralizeddevice can be employed for post processing. The post processing caninclude creation of thumbnail images for enhanced navigation. Forexample, the scene/slide detection technique can enable a user to saveindividual images from captured lecture files every time a slide changeoccurs, thereby creating thumbnail images with appropriate time-stampinformation. The thumbnail images can also be used in conjunction with aplayer module for navigation of captured video. During navigation, thelocation of the videos, the output path, and the media server URL forboth ASX and RPM files can be specified.

Presentation/lecture files can be created from captured session filesbased on “in” and “out” markers that have been confirmed using theeditor application and by creating thumbnails via the scene/slidedetection technique. Additionally, identified slides (i.e., videosegments) can be replaced with placeholder slides that can be selectedusing an administrative tool in accordance with an embodiment of thepresent disclosure.

The centralized device can run unattended and process captured mediaassets, which may include session files, presentation/lecture files, orthe like, as the files are received or pass through production workflow.The centralized device can synchronize with a database via a Web serverand can run concurrently on several computers for faster processing andturnaround time at a session/venue.

FIG. 24 depicts an exemplary user interface 2400 for a serverapplication that can be executed on a centralized device according to anembodiment of the present disclosure. The server application can beemployed for rapid enhancement and post processing of captured contentreceived from the capture and/or editor applications, and forsynchronizing metadata between applications (e.g., the administrativetool, the capture an editor applications, a kiosk tool, a productionwebsite, etc.).

The centralized device can primarily exchange data with individualcapture stations to retrieve the captured media assets and to split outindividual presentation files from the captured sessions. For example,as shown in FIG. 24, once the import of captured media assets isinitiated, the server application can search for and display identifiedrecorded sessions in the “Process Folder” window 2405. Successfulimports can be listed in both the “Completed Folder” 2410 and the“Import Log” 2415.

By selecting an “Autosave XML” checkbox 2420, an up-to-date event datafile (e.g., “conferencedata.xml”) can be created at a certain interval,which can be used by both the capture and editor applications. Automaticsplitting of imported media assets can be initiated by selecting a“Start Splitting” button 2425. The server application can then monitorfor presentation segments for which markers have been confirmed (i.e.,the markers can be set during capture using the exemplary captureapplication shown in FIG. 22 and confirmed using the exemplary editorapplication shown in FIG. 23). When the markers for a presentation areconfirmed, the server application can split out that particularpresentation file from the media assets captured for the entire session.After a being split out, the presentation's status can be changed (e.g.,to “Ready for Quality Control (RFQ)”).

4) Enhanced Search and Retrieval

Optionally, the scene/slide detection technique can be used inconjunction with OCR software to create time-stamped files containingterms, which can enhance search and retrieval of the captured mediaassets. Additionally, OCR can be applied to the visual aid imagesdisplayed during the presentation, and speech recognition can be appliedto the audio data recorded during the presentation to enabletranscription of the words spoken when the visual aid images werepresented. In this way, the entire presentation can be searched. Forexample, a user can search transcripts of the visual aid images and thepresenter's spoken words to locate particular topics of interest.

The scene/slide detection technique can also be used to create thumbnailimages, which can be used to provide users with visual cues of lecturecontent, thereby enabling rapid search and retrieval of capturedpresentations/lectures.

5) Rapid Review of Materials

FIGS. 25-28 depict exemplary user interfaces for managing capturedpresentations using a web-based administrative tool according to anembodiment of the present disclosure. In addition to managing lecturescaptured for a conference, the administrative tool can also integrate aworkflow that enables, among other functions, quality control oflectures after thumbnail images have been generated, editing of lecturesby replacing selected slides with placeholder slides/video clips, whichcan be inserted at the correct location via the server application, andremoval of unwanted thumbnail images when extra thumbnails aregenerated. A user can also use the administrative tool to alter, on aper-lecture basis, parameters used for scene/slide detection and cansubsequently re-run scene/slide detection when a particular lecturerequires adjustment.

FIG. 25 depicts an exemplary user interface 2500 for the administrativetool according to an embodiment of the present disclosure. As shown inFIG. 25, the administrative tool can be implemented as a web-based toolfor managing captured presentations/lectures, as well as specifyingparticular lectures for review, quality control, and editing prior toproduction. The administrative tool can also be used to enable rapidvisual review of captured presentation media by employing scenedetection algorithms to detect scene changes. The administrative toolcan also enable an end-user to review the captured presentation contentand make substitutions and/or deletions, as necessary.

6) Removing Slides and Inserting Visual Information

The administrative tool can be configured to manage the processedcaptured media assets to produce modified presentations. For example, itmight be desirable to remove particular captured presentation content(e.g., content which might be published in journals, or content forwhich the speaker might not have copyright permission to reproduce involumes that are to be sold as part of an electronic collection createdusing the techniques described herein). In one embodiment, theadministrative tool can be configured to enable modification of thevisual aid images that are identified by image data markings set duringcapture using the capture application.

FIG. 26 illustrates an exemplary user interface 2600 for theadministrative tool that enables a user to edit a presentation byselecting a radio button 2605 to replace selected slides withplaceholder slides/video clips and delete unwanted thumbnail images whenextra thumbnails are generated. FIG. 27 illustrates an exemplary pop-upwindow 2700 that can be displayed when the user selects the “ReplaceWith” option for a particular slide. As shown in FIG. 27, the user canthen select placeholder slides/video clips from a folder of placeholdercontent with which the selected slides can be replaced. For example, asshown in FIG. 28, the user selected “Slide 5” 2805 for replacement withan image of the presenter 2810. Other replacement content can be used toreplace selected images, such as advertisements, presentation previews,information related to presentations, and information related topresenters, etc.

When a slide is selected for replacement or deletion, a rapid“stitching” technique for editing captured presentation files can beused to change video content without the need for re-rendering thevideo. Stitching can be used to eliminate the typical burden of videoediting and re-rendering and re-compiling the edited files into a newdigital media file. For example, the administrative tool can be used todetect key-frames in a video stream and enable an end-user to select avideo sequence that can be substituted for a scene to be deleted. Usinglow-level C code, or other low-level code, the administrative tool canstitch the video files together at edit points, thereby eliminating theneed for re-rendering the video and reducing the time required to edit acaptured video stream to seconds (typical video editing applications canrequire up to a 1:1 time ratio between duration of a video and amount ofrendering time required to process and produce a final edited product).The stitching technique can also preserve audio associated with thevideo sequence.

7) Volume Division and Designating Output Means

The administrative tool can also be used to divide capturedsessions/conferences into logical segments, and to publish the segmentsfor various media types, which can include physical and/or on-linedistribution media. In an embodiment, the distribution media can beconfigured to incorporate e-commerce functionality, such aspay-per-view, sponsorship-based payment, subscription models, andinstitutional sales, among others.

8) Replication Enhancement

When physical media replication is desired, a production tool can beused to enable rapid creation of digital media, which can be transferredand reviewed on-line prior to replication on or off-site. Using theaforementioned techniques and tools, replication can be achieved rapidly(e.g., within minutes) following a presentation.

In one embodiment, the production tool can be configured to format themodified presentations of at least one session/conference fordistribution on distribution media. For example, the production tool canbe used to divide the conference into different logical volumes ortracks. The production tool can also be used to specify desiredpublication media (e.g., on-line, DVD/CD, etc.), as well as includenavigation functions and other media associated with particularpresentations, such as abstracts, articles, or the like, which can havevarious formats, including PDF format.

FIG. 29 depicts an exemplary user interface 2900 of a production toolfor publishing captured lectures according to an embodiment of thepresent disclosure. In the example embodiment of FIG. 29, the productiontool is configured to master DVDs on-site at a conference by executingthe following general steps. First, the production tool can calculatethe number of discs needed to accommodate the content to be published tothe discs and can be configured to automatically span several discs ifneeded. Next, the production tool can copy the captured media assets tobe included on a specific disc to a corresponding folder (e.g., a “DVDMaster” folder). Finally, the production tool can dynamically generatenavigation pages and copy common files needed for the disc. These stepscan be implemented in accordance with storage requirements andorganizational structure for publishing to media types other than DVDs,such as podcasts, MP-3, CDs, flash memory, magnetic memory, etc.

Additionally, sponsorship information can be created in the form oflogos, animations, audio files or the like, and incorporated into thepublished media files, and/or the media player format (e.g., bannerads), and/or media labeling. Sponsorship information can also becustomized on a per volume basis or per other logical divisions ofcontent (such as the production of discs for individuals or groups ofparticular individuals). Optionally, the production tool can be used toenable custom distribution media creation by end-users. For example,end-users can browse a library of captured content to select desiredcontent for distribution and can also select a desired media type(s).

In one implementation, the production tool can include file systeminformation such that the distribution media types are compatible withdifferent operating systems. For example, software running on Apple,Inc. computers such as Toast® (roxio.com/en/products/mac_products.jhtmlon the World Wide Web) can be used to create hybrid distribution mediathat includes unique contents for Mac or PC users. Likewise, softwaresuch running on computers equipped with the Windows operating systemsuch as TransMac (asy.com/scrtm.htm on the World Wide Web) can be alsobe used to produce such hybrid distribution media.

After the production tool generates an image of the captured mediaassets, the image can be replicated for large-scale distribution.Replication can be accomplished using a variety of devices, includingrobotic replication machines such as the Rimage (rimage.com/home.html onthe World Wide Web), which can print labels on physical distributionmedia, as well as replicate the content. Additionally, replication canbe accomplished using pre-printed blank physical distribution media,which can be replicated using systems such as the NEC-based replicationtower (cddvdking.com on the World Wide Web).

9) Combining on-Line Testing and Certification

A testing and certification tool, such as the Net.SCORE™ applicationavailable from Astute Technology, can be configured to combine on-linetesting with on-line lectures for integration of learning objectives,e-commerce, testing and on-line certification. For example, thehardware/software capture platform 2105, shown in the embodiment of FIG.21, includes a Net.SCORE web-based testing and certification tool 2140.

The testing and certification tool can be implemented to enable on-linetranscript generation, as well as maintenance of a log of live eventattendance credits versus on-line or enduring (i.e., hardcopy) materialcredits. The maintenance of such a log can help credentialing entitiescomply with regulations by preventing an award of double credit for thereview and completion of a course that is available in both on-line andlive formats. The testing and certification tool can also be implementedto enable a presenter to identify weaknesses of attendees prior topresenting a lecture. In this way, the presenter can customize thelecture prior to presenting it to specifically address the identifiedweaknesses, if desired.

10) Exemplary Method

FIG. 30 illustrates an exemplary method 3000 for enhanced capture,management and distribution of live presentations. Not all of the stepsof FIG. 30 have to occur in the order shown, as will be apparent topersons skilled in the relevant art(s) based on the teachings herein.Other operational and structural embodiments will be apparent to personsskilled in the relevant art(s) based on the following discussion. Thesesteps are described in detail below. It should be noted that, inaccordance with an aspect of the present disclosure, the methodillustrated in FIG. 30 can be employed in conjunction with acomputer-based system, where the method can be implemented in hardware,software, firmware, or combinations thereof.

In step 3005, media assets of live presentations comprising a sessionare captured. The media assets can include image data of sequentiallypresented visual aids accompanying the live presentations and audiodata. Exemplary visual aids include slides, photographs, graphs,discrete motion picture clips, and text. The capturing includesreal-time image data marking of the image data for identification ofindividual images and session marking of the image data for demarcationof individual presentations of the session.

For example, a capture application, such as the capture applicationillustrated in FIG. 22, can be executed on a capture device to enable auser to mark the image data for identification of individual images(e.g., by setting “random” markers) and for demarcation of individualpresentations of the session (e.g., by setting “in” and “out” markers).Optionally, the administrative tool illustrated in FIG. 26, can also beused to mark the captured image data for identification of individualimages (e.g., by selecting the “Replace With” or “Delete” radio buttons2605, as shown in FIG. 26). Step 3005 can also include synchronizing thecaptured media assets for each presentation by generating time-stampdata.

Optionally, method 3000 includes the step of editing the captured mediaassets, including modifying the image data/and or session markings for apresentation and confirming the markings for each presentation. Forexample, in one implementation, an editor application, such as theeditor application illustrated in FIG. 23, can be executed on thecapture device or on the centralized device to enable a user to modifyand confirm the markings set for each presentation.

In step 3010, the captured media assets for the session are processed.The processing includes automatically dividing the captured media assetsfor the session into discrete files associated with the individualpresentations based on the session markings. For example, a serverapplication, such as the server application illustrated in FIG. 23, canbe executed on the centralized device for rapid enhancement and postprocessing of captured content received from the capture and/or editorapplications. Additionally, to enable text searching of thepresentations for the session, step 3010 can include applying opticalcharacter recognition to the image data of the visual aid images, and/orapplying speech recognition to the audio data to enable transcription ofthe spoken words during the presentation. In one implementation, step3010 can include analyzing the image data to automatically generatethumbnail images for each presentation that enable navigation of thepresentations. In this case, the analyzing can include detecting changesin sequences of visual aid images to mark the image data.

In step 3020, the processed captured media assets are managed to producemodified presentations. The managing includes modifying the visual aidimages identified by the image data markings. For example, anadministrative tool, such as the web-based administrative toolillustrated in FIG. 24, can be configured for managing capturedpresentations/lectures, as well as for specifying particular lecturesfor review, quality control, and editing prior to production. In oneimplementation, step 3020 can include deleting the visual aid imagesidentified by the image data markings to produce the modifiedpresentations. In another implementation, step 3020 can includereplacing the visual aid images identified by the image data markingswith replacement image data to produce the modified presentations. Inthis case, the replacing can include stitching together the image dataand the replacement image data.

In step 3025, the modified presentations of at least one session areformatted for distribution on distribution media. For example, aproduction tool, such as the production tool illustrated in FIG. 29, canbe configured for publishing captured presentations/lectures. In oneimplementation, step 3025 can include formatting the modifiedpresentations for at least one session for on-line media distribution.In another implementation, step 3025 can include formatting the modifiedpresentations for at least one session for physical media distribution.In this case, the formatting can include dividing the formattedpresentations for the at least one session into logical units when theformatted presentations span multiple physical media.

Additionally, step 3025 can include generating mechanisms for navigatingthe formatted presentations on the distribution media. Step 3025 canalso include enabling end-user selection of desired formattedpresentations for distribution on the distribution media. Step 3025 canfurther include incorporating customized sponsorship information intothe formatted presentations for distribution on the distribution media,as well as incorporation of additional media associated with theformatted presentations (e.g., abstracts, articles or the like) fordistribution on the distribution media.

Optionally, the method 3000 can include the additional step of trackingcredits awarded to an attendee for completing a course by attending alive presentation to prevent awarding additional credit to the attendeefor completing the same course on-line or for completing the same courseusing hardcopy materials. In another embodiment, the method 3000 caninclude the additional step of identifying weaknesses of attendees priorto delivering a presentation to customize the presentation in accordancewith the identified weaknesses. For example, a testing and certificationtool, such as the testing and certification tool 2140 illustrated inFIG. 21, can be configured to integrate on-line testing andcertification.

CONCLUSION

Techniques consistent with the present disclosure provide, among otherfeatures, rapid conversion of live presentations into electronic mediaand effective management of captured media assets for distribution. Theforegoing description of an implementation of the invention has beenpresented for purposes of illustration and description. It is notexhaustive and does not limit the disclosure to the precise formdisclosed. Modifications and variations are possible in light of theabove teachings or may be acquired from practicing of the disclosure.The scope of the invention is defined by the claims and theirequivalents.

1. A system for capturing and distributing presentations, comprising: atleast two capture devices configured to capture media assets of livepresentations comprising a session, the media assets including 1) imagedata of a plurality of sequentially presented visual aids accompanyingthe live presentations, 2) audio data, and 3) video data of a presenterof the corresponding live presentation, wherein at least two of thevisual aids are selected from the group of images consisting of slides,photographs, graphs, discrete motion picture clips, and text, andwherein each capture device includes an interface that enablesreal-time 1) image data marking of the image data for identification ofindividual images and 2) session marking of the image data fordemarcation of individual presentations of the session; a centralizeddevice configured to process the captured media assets from each capturedevice, wherein the centralized device is configured to automaticallydivide the captured media assets for the session into discrete filesassociated with the individual presentations based on the sessionmarkings; an administrative tool configured to manage the processedcaptured media assets to produce modified presentations, wherein theadministrative tool enables modification of the visual aid imagesidentified by the image data markings; and a production deviceconfigured to format the modified presentations of at least one sessionfor distribution on distribution media.